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ABSTRACT 


Gesture based communication system is designed keeping in mind the 
communication needs of the people who are either bedridden or suffer from 
some kind of speech impairment. The main benefit of this system is that it's 
not necessary for some other person to be present with the bedridden person 
always by providing message services. It is a microcontroller based system 
that uses buttons present on the glove to display (on a screen) and play a 
particular message (through speaker) that is already stored in the system and 
uses joystick to compose a message not present in the system and to send the 
same to a person not physically there. The person can also communicate with 
the disabled person through a software to ensure 2-way communication. In 
this way, this project will help to lower the communication gap between 
poorly abled and normal people. 
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I. INRODUCTION 

In the recent years, there has been a rapid increase in the 
number of hearing impaired and speech disabled victims due 
to birth defects, oral diseases and accidents. We have also 
witnessed an increase in the number of semi-paralyzed and 
bedridden people. While speech impaired people can 
communicate with one another using sign language but for 
the normal people it is difficult to understand. Thus, it 
becomes difficult for such people to convey their message 
within the society. While many systems have been devised 
for them, but in all the cases the normal person has to be 
physically present there. 

Also, in case of people who are bedridden due to old-age or 
some accident, they always need a person with them in case 
they might need anything. But in today's fast track world, it 
is not possible for a person to stick to a bedridden person. 
Thus, it is required to design a system for fulfilling their 
communication needs as well. 

Keeping in mind the above mentioned problems, we have 
devised a system to overcome them. Gesture Based 
Communication System is a microcontroller based system 
that uses buttons present on the glove to display and play a 
particular message and uses joystick to compose a message. 
It also provides messaging service, in case the person is not 
present there. The other person can also communicate with 
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the poorly abled person using a software which displays the 
message on a screen. This is to ensure 2-way 
communication. It uses the following devices: Arduino mega 
2560, glove with buttons on each of its fingers, GSM 900A 
module, speaker, MP3 module, Bluetooth module, analog 
joystick and a 16x2 LCD. The buttons have a pair of wire 
with one grounded and the other wire is connected to the 
Arduino. The GSM module, Bluetooth, joystick and LCD are 
also connected to the Arduino and the joystick is also 
connected to the LCD to form or send messages. Using the 
GSM module the message is sent a specific person and he can 
respond using an open source software connected through 
Bluetooth to the system. 

II. SYSTEM MODEL 

The buttons on the glove, whose one end is connected to the 
ground on PCB and the other is connected the Arduino, are 
used to give various commands. The Arduino is interfaced 
with mp3 in such a way that the recordings in memory card 
of mp3 player will be played according to the conditions and 
the output will be produced through the speaker. The 
Arduino communicates serially with the mp3 player as RX 
and TX pins of Arduino are connected to TX and RX pins of 
mp3 player respectively. MP3 player is controlled by the 
Arduino, that is, Arduino decides the volume, which 
recording is to be played, etc. but does not do the work itself. 
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Fig.l: Block diagram of overall system 


The Arduino is programmed in such a way that it does not 
control the mp3 player when the initial recording is played 
in which we play the following recording "Gesture controlled 
communication system by Heebatullah, Zarka Latief, Nazish 
Ara, Sumyra Jan and Bazila Nazir of SSM College of 
Engineering and Technology" and the same content is 
displayed on the LCD. Depending upon which button is being 
pressed the associated action takes place. 

Also a GSM module is interfaced with the Arduino. This GSM 
module helps the disabled person to communicate with the 
person who is not physically present there, i.e., the disabled 
person can communicate with him through SMS. Also a 
joystick is connected to the LCD and interfaced with the 
Arduino so that the disabled person can write his own 
message. 

III. WORKING 

Consider a situation where a person is wearing this glove. 
First, a battery is connected with the circuit the switch on the 
PCB is turned which in turn activates and connects all the 
devices in the circuit with the Arduino. The name of the 
project is also displayed on the LCD and the default 
recording is played through speaker. When a button is 
pressed, the pre-recorded message is played through 
speaker and the message is also displayed on the LCD, e.g. 
button 4 is pressed on the glove, the following recording is 
played: "I am hungry, give me something to eat". Also the 
following message is displayed on the screen: 



^ m TRti.Q 

090703 62176 ^ l 



Vr'udr^ruilsy. Jan, 17 - 1 3:££ 


lam huingft: Oitf&-rnS- Something 


Fig-3: 


Also if the person wants to write his own message, he can do 
so with the help of joystick. The joystick can be moved in 4 
directions (left, right, up, down) and these corresponding 
actions are performed: select alphabet, add space, change 
alphabet and send message. With the help of joystick the 
person can compile his own message. 


The image displayed below, shows a sentence that is created 
by the user himself using the joystick the movement of 
joystick is described follows: 



Fig-4: 


By default "a" is displayed on the LCD. When joystick is 
moved in the downward direction, the alphabet changes on 
the LCD. When joystick is moved in the left direction, the 
displayed alphabet is selected and put into the word. When 
joystick is moved in upward direction, space is added in the 
composed sentence. When the joystick is moved in the right 
direction, the so-formed message is sent to the person whose 
no. is already fed into the system. Fig.3 shows the message 
when sent to the person whose number is fed into the 
system. Thus, joystick helps the person to push the 
boundaries, i.e., forms his own message. 


CONCLUSION 

This project will fulfill the communication needs of people 
who have speech or hearing impairment either by birth or 
due to some accident or operation or who are bedridden. 
While many systems have been devised for them, but in all 
the cases the normal person has to be physically present 
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there. Also, in case of people who are bedridden due to old- 
age or some accident, they always need a person with them 
in case they might need anything. Thus, to fulfill the needs of 
such persons we have designed this system. This system 
uses buttons present on the glove to display and play a 
particular message and uses joystick to compose a message. 
It also provides messaging service, in case the person is not 
present there. The other person can also communicate with 
the poorly abled person using a software (through 
Bluetooth] which displays the message on a screen. This is to 
ensure 2-way communication. In this way this project will 
help to lower the communication gap between disabled and 
normal people. 
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FUTURE SCOPE 

Since in this project we have used a joystick to compose a 
message of our own, we can also use a speech-generating 
device or voice output communication aid to convert this 
written sentence into a voice. This can be a special device 
that supplements and replaces speech/writing for people 
who have problems communicating in a traditional way. 

The system can also be made compatible with different 
languages for different users. 


We can also be connected with the internet to check emails, 

browse the net, etc. 

Also we can use finger-touch device rather than the joystick. 

We can also introduce a system using which the normal 

person can message the poorly abled person and the same 

message can be heard using speaker. 
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